Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3.8] bpo-42230: Improve asyncio documentation regarding accepting sets vs iterables (GH-23073) #23105

Merged
merged 1 commit into from
Nov 2, 2020

Conversation

jstasiak
Copy link
Contributor

@jstasiak jstasiak commented Nov 2, 2020

People call wait() and as_completed() with various non-set iterables,
a list should be the most common but there are others as well[1].

Considering typeshed also documents wait()[2] and as_completed()[3]
as accepting arbitrary iterables I think it's a good idea to document
the status quo better.

[1] aio-libs/aiokafka#672
[2] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyi#L161
[3] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyi#L40.
(cherry picked from commit 3d86d09)

https://bugs.python.org/issue42230

…ts vs iterables (pythonGH-23073)

People call wait() and as_completed() with various non-set iterables,
a list should be the most common but there are others as well[1].

Considering typeshed also documents wait()[2] and as_completed()[3]
as accepting arbitrary iterables I think it's a good idea to document
the status quo better.

[1] aio-libs/aiokafka#672
[2] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyiGH-L161
[3] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyiGH-L40.
(cherry picked from commit 3d86d09)

Co-authored-by: Jakub Stasiak <jakub@stasiak.at>
@asvetlov
Copy link
Contributor

asvetlov commented Nov 2, 2020

Thanks for the manual backporting!
Python 3.6 and 3.7 are in secure mode, they don't accept such changes

@jstasiak
Copy link
Contributor Author

jstasiak commented Nov 2, 2020

No worries, understood!

@asvetlov asvetlov merged commit ad37c66 into python:3.8 Nov 2, 2020
@jstasiak jstasiak deleted the backport-3d86d09-3.8 branch November 12, 2020 11:04
@kklingenberg
Copy link

Hello. I found that the documentation is still slightly misleading, that or the implementation is: if given generator-based iterables to asyncio.as_completed a TypeError is raised. I prepared this simple test:

import asyncio


async def sleep_job(n):
    await asyncio.sleep(n)
    return n


async def test():
    print("with lists")
    for job in asyncio.as_completed([sleep_job(n) for n in (4, 2, 3)]):
        print(await job)

    print("with sets")
    for job in asyncio.as_completed({sleep_job(n) for n in (4, 2, 3)}):
        print(await job)

    print("with generators")
    # This raises: TypeError: expect an iterable of futures, not generator
    for job in asyncio.as_completed(sleep_job(n) for n in (4, 2, 3)):
        print(await job)


if __name__ == "__main__":
    asyncio.run(test())

Maybe this is the intended behaviour, in which case the documentation could use some sort of warning. But I'm not sure the restriction on generators is intended.

¿What do you think? ¿Should I report this as a documentation bug?

@jstasiak
Copy link
Contributor Author

My asyncio background is severely lacking, the relevant part of the code seems to be producing a false positive here:

    if futures.isfuture(fs) or coroutines.iscoroutine(fs):
        raise TypeError(f"expect an iterable of futures, not {type(fs).__name__}")

On the other hand I vaguely recall there may be a historical reason for this, someone more familiar with asyncio would have to provide some context.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants